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Using NLP to Define the Scope for Stakeholder 
Assessment of Simulated Service Qualities 


The paper is devoted to defining the scope of research activities aimed at involving business stakeholders in 
a software process in a form of assessing the perceived quality of the service-oriented system in its usage 
context when the initial specification of the system is available in natural language form. We propose to use 
NLP techniques to extract the scope from this specification and to represent it in the format of specific 
predesign models compatible with the rest of the simulation solution. 


Introduction 


Gathering the opinions of business stakeholders (not necessary possessing any IT 
background) is considered an important part of the software process by both software 
engineering researchers and practitioners. The main subject of such opinions is currently the 
prospective system’s functionality; it forms the foundations for widely studied functional 
requirements elicitation problem. Less studied but still important kind of opinions are 
assessments of the quality of the prospective system. The reason of this importance is that if 
stakeholders cannot express such opinions early in the software development lifecycle these 
they could be easily lost. As a result of this loss, the understanding of the desired quality of 
the system becomes biased towards the view of the IT people. This could lead to the 
stakeholder dissatisfaction with the quality of the software under development (SUD) late in 
the development lifecycle. This is dangerous, as the problems with satisfying the customer 
revealed that late could easily lead to the complete failure of the whole project. 

To address this problem, we proposed ISAREAD-S framework (/nteractive Simu- 
lation-Aided Requirements Engineering and Architectural Design for Services) [1], [2]. It is 
aimed at investigating ways of supporting stakeholder involvement in the software process 
by allowing business stakeholders to assess the perceived qualities of the prospective system 
in its usage context. To implement such support we plan to elaborate a set of simulation- 
based methods aimed at making QoS (quality of service) assessment procedures accessible to 
the business stakeholders and using their assessments as a driving force for activities related 
to requirements engineering and architectural design. 

The research goal is to define a scope for these simulation activities (SAREAD-S 
scope), i.e. the initial set of services and qualities of interest together with possible usage 
contexts. We propose to obtain this scope by analyzing the existing description of the problem 
domain by means of Natural Language Processing (NLP) techniques. 

The structure of the paper is as follows. Section 1 shows the principles of the existing 
procedures (mechanisms) for organizing the interaction with stakeholders; these mechanisms 
form the target activities for the scope definition solutions proposed in the paper, Section 2 for- 
mulates the problem statement, Section 3 describes the models used to define the ISAREAD-S 
scope, Section 4 outlines the proposed NLP-based approach for transferring the specification 
for the SUD or the problem domain into the ISAREAD-S scope definition, and finally sec- 
tion with necessary conclusions. 
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1. Interactive assessment of simulated service qualities 


ISAREAD-S approach includes three kinds of mechanisms. 

Service-level mechanisms organize assessments for the simulated qualities of the 
particular services. We define two mechanisms of this kind: model composition and model 
execution mechanism. Model composition mechanism combines parameterised simulation 
models with corresponding interaction models for the particular qualities to form interac- 
tive simulation models allowing business stakeholders to participate. The participation 
depends on the quality being processed; it involves experiencing the quality and making an 
assessment using specific scale. Model execution mechanism is based on model driven 
paradigm as it treats interactive simulation model as executable model; it is responsible for 
interactive execution of such models to organize assessments of the qualities of interest. 

Process-level mechanisms put these service-level assessments into the context of simu- 
lated usage processes. We propose to represent such usage processes using Business Process 
Modeling (BPM) notations such as BPMN or Petri Nets. Similarly to the service-level case, 
process-level model composition and model execution mechanisms are defined. The first me- 
chanism embeds service-level interactive simulation models for the services of interest into the 
business process model to form interactive simulation model for service usage process. Model 
execution mechanism executes this model to establish a user session. In the course of this ses- 
sion, the user is asked to perform assessments of the qualities of interest for the services of interest. 

An iterator mechanism controls the interaction sessions and gathers all necessary 
assessments for the particular SUD (treated as a set of services of interest). 

On top of these mechanisms the policies for requirements elicitation, negotiation, and 
validation, architecture assessment and lifecycle support are defined. In addition, the 
adapters are responsible for the transformation of the external information related to the 
project (such as the amount of available resources) into the numerical values for the factors 
influencing the simulations. 


2. The problem of ISAREAD-S scope definition 


The above description of the ISAREAD-S framework misses one important issue: it 
is not clear how to define the set of services and qualities of interest and possible usage 
context (which form the scope of the future simulation-related activities) prior to the asses- 
sment. It was implicitly assumed that this set is known beforehand to software engineers. 
Unfortunately, it is not always the case; as a result, defining such set by hand every time 
the new project begins could be cost-ineffective. 

The ways of handling this problem vary significantly depending on availability of the 
description of the prospective system (or at least the problem domain). In this work, we 
restrict ourselves with the case when such description is available in natural language form, 
which leads to the following research question which determines the problem statement: 
How to determine the set of services and qualities of interest together with possible usage 
contexts from the existing natural language specification of the prospective SUD and make 
it available to the ISAREAD-S framework? 

To address this question, we propose to use Natural Language Processing techniques 
to process the specification documents with a goal of obtaining structural representation of 
the scope of the ISAREAD-S application (Fig. 1). This way, such process could be auto- 
mated which reduces the costs of applying the framework. 


3. Representing ISAREAD-S scope with predesign models 


Before investigating possible ways to process the description of the prospective system or the 
problem domain we need to define the target of this processing, namely, the format we are plan- 
ning to use to represent the scope of the ISAREAD-S framework to be used by its components. 
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Figure 1 — Problem definition 


As our framework focuses on supporting stakeholder involvement into the software 
process activities we should implement the interface which, on the one hand, is suitable for 
validation by business stakeholdera(qq. the people without special Seneuea n the other 

Service Hed 


hand, is suitable for implementing! t see for the simulation o ies. We use 
special requirements models for .t fie These models prbBOBEMERES a tabular 
form of ee acitie Ate f¥Spmain expertsgentences 


To implement us Pech view of the SUD functionality (functional ISAREAD-S 
scope), i.e. the set of services of interest, we use Klagenfurt Conceptual Predesign approach 
developed in Klagenfurt University [3], [4] which defines a metamodel (Fig. 2) for Klagen- 
furt Conceptual Predesign Model (KCPM). This model consists of static and dynamic parts, 
which include process-invariant and service process information respectively. Thing-type and 
connection-type form the static part. Thing-type generalizes the class notion as well as 
attribute and value notions. The examples of this concept are persons, objects, resources, at- 
tributes, characteristics, abstracts and other entities. Connection-types represent relations bet- 
ween thing-types. Operation-type and cooperation-type constitute the dynamic part. Ope- 
ration-types are intended to define service operations, their actors and parameters, which are 
expressed by thing-types. Cooperation-types are used to model business processes which 
orchestrate services. A cooperation-type consists of a triple of sets <Prc, {A,O}, Poc>, where 
Pre — the set of pre conditions, Poc — the set of post conditions, {A,O} — the set of pairs con- 
sisting of an operation-type and an actor executing this operation. 


Figure 2 -KCPM metamodel for a functional ISAREAD-S scope 


To implement user-centered view at the qualities of the prospective SUD (non- 
functional (quality-related) ISAREAD-S scope), i.e. the set of qualities of interest, we use 
Quality-Aware Predesign Model for Services (QAPM-S) [5]. It is connected to KCPM (fun- 
ctional model) via common abstract concept ModelingElement and dynamic elements — 
operation-type and cooperation type (Fig. 3). This model is based on the notions defined in 
ISO/IEC 9126 standard such as quality category, quality characteristic, quality metric, etc. 
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Figure 3 —- QAPM-S metamodel for a quality-relatpgd AsREAD-S scope 
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These acts are aes iis using different LR meth & ” For theaplgeat we use one 
of the approaches based on context-free grafimars apybfepgmsky po MRR erTHS More 
detail information is described in previous paper [7]. This approach co Phistic 
part-of-speech tagging with sophisticated rule-based chunking and produces from free 
requirements text structured ontology-oriented tree-based output, available in XML format [4]. 
Its main benefits are: MeasurementUnit - : 

— enriching classigal part-of-speech (POS) tagging wiualityRequirement 

— compound GetIAWOR cation related to term identification; 

— two-level vpryryyeclass identification (firstly tA@@RNPAratOlnt the number of 
noun pseudo-objects (nPO); secondly after chunking procedure correction according to the 
verb arguments number); 

— post-modifier of nouns; 

— flexible settings allowing to manipulate with different lexical features (phrasal 
nodes, etc.); 

— verb roles disambiguation. 

But in spite of these advantages there are some difficulties with the processing of 
complex language constructions and English language dependency. To process other lan- 
guages the developed engine requires the appropriate language resources both the language 
data itself and its structure rules information. For increasing accuracy of text processing 
and targeting an extended set of human languages more flexible and semantic-oriented 
approaches (like MTT [8], or UNL [9]) can be embedded in ISAREAD-S framework. 

The example of this requirements elicitation step is listed below: 

- <sentence> 
- <sentence type="subordinate"> 
<con0 derivedPOS="n0" type="subord">If</con0> 
= 
<det0 form="general" type="def">the</det0> 
<n0 base-form="order" derivedPOS="v0" num="sg" type="common">order</n0> 
</n3> 
<v0 base-form="come" form="ind" num="sg" phrasalverb 
temp="pres" verbclass="iV">comes</v0> 
<p0 derivedPOS="pt0" phrasalverb="come in">in</p0> 
</sentence> 


—N" 


come in" ps="3" 
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-<n3> 
<det0 form="general" type="def">the</det0> 
- <n0 desc="compound" type="common"> 
<n0 base-form="bookkeeping" corelex="act" derivedPOS="a0" num="se" 
type="common">bookkeeping</n0> 
<n0 base-form="department" corelex="grs" num="se" 
type="common">department</n0> 
</n0> 
</n3> 
<v0 base-form="check" derivedPOS="n0" form="ind" temp="pres" num="sg" ps="3" 
verbclass="tvag2">checks</v0> 
==n3> 
<det0 form="general" type="def">the</det0> 
<n0 base-form="payment" corelex="poa" num="sg" type 
<eng 
</sentence> 

The requirements sentence “Jf the order comes in, the bookkeeping department checks 
the payment” was transformed into structured tree-based head-lexicalized format. The second 
part of sentence is governed by verb “check” which has two arguments: subject — noun 
phrase “the bookkeeping department” and object — noun phrase “the payment”. This XML 
material is ready for interpretation and modeling notions extraction. 

Interpretation. After free text processing the obtaining output is ready to be inter- 
preted into different modeling concepts and parameters. For modeling context-aware 
functional service aspects the user-centered approach is used [3]. As this approach’s pivot 
is the stakeholder involvement in development process, its main view is the set of 
glossaries (organized into QAPM-S model [5]), which are filled with modeling concept 
from processed text. For this purpose we use various interpretation rules, mainly based on 
predicate argument structure of verbs and its agentivity. The list of verb classes identified 
during the linguistic processing and used for interpretation can be found in [7]. The head of 
noun phrases and the parts of compound nouns are transferred into thing-type glossary. The 
containing and attributive relations create entries in connection-type glossary. The 
transitivity of verbs flags to what element of behavior model part it belongs to. For 
example, verbs with categories “tvag2” (monotransitive verb with agent subject), “tv3” 
(ditransitive verb), “1V” (intransitive verb) [1] denote an operation-type. 

The set of such operation types correspond to the set of services of interest 
comprising the ISAREAD-S scope. The noun on subject place is mapped to an executing 
actor, whereas on object place to a calling actor or parameter depending on context and 
other trigger rules. The verb without agent subject, e.g. “tv2”, denotes the condition in the 
cooperation-type glossary [10]. 

Quality characteristics being modeled are structured into the QAPM-S model and can 
also be extracted from the NLP output. The quantor phase contains the quantitative quality 
characteristics in this model. The service usage context represented by operation-type and 
cooperation type used from its original glossaries is also interpreted using sentence and 
phrase relationships. 

The main rules of interpretation are listed in tables | — 2. The example of interpre- 
tation engine applying for the mapping from tree-based structured requirements into 
glossaries’ entries is represented in fig. 4. The engine process the requirements sentence 
applying step by step the interpretation rules climbing up the rule order. 

Consider more detail the list of rules used to form functional requirements model. Rule / 
reflects the basic principle that every noun is thing or abstract notion and therefore mapped 


—" 


common">payment</n0> 
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into thing-type entry. The attribute assigned during noun type identification can be trans- 
formed to “classification” field in thing type. Rule 2 means that in the case of compound, i.e. 
congruence between parts of compound concerning proper or common type. Rule 3 denotes 
the relation between the head and dependent member in compound noun structure. Rule 4 
identifies the post-modifier in the case of prepositional phrase (p2) [11]. “p2” can play role 
both as the argument of verbal phrase and as adjunctive or adverbial modifier of other 
sentence argument. This ambiguity is tried to solve by developed approach [6]. And in the 
case of modifier it should reproduce the entry in connection type glossary. Rule 5 captures the 
abstraction generalization and component/object which are treated as connection types [12]. 
These cases occur if the verb “to be” in 3™ person functions as the main verb in the sentence. 
Rule 6 is intended for possession relation between subject and object. It occurs then the verb 
“to have” functions as the main verb. Other default cases for the verb with more than one 
argument are applied by Rule 7. These cases present the relation between verb arguments 
with respect to its valency. Rule 8 provides the operation-type identification. The noun phra- 
se, which is agent subject, becomes an “actor” whereas other noun phrases, which have other 
argument roles, are mapped into parameters of operation-type. Rule 9 is executed in the case 
of past participle of transitive and intransitive verbs and denotes the pre-condition in 
cooperation-type glossary. Rule 10 specifies the condition for operation-type if the verb is 
not an agentive verb and all the arguments of the verb become candidates for involved thing- 
types (according to the procedure described in [11]). Rule J] indicates the conditions of 
property/state type; it means then adjective or adverbial phrase describe the properties of 
thing-type. Rule 12 decodes the event type of condition; it means then act is carried out, but 
the subject does not act. Rule 13 allows us to suppose the passive voice construction as the 
candidates for conditions of cooperation-types. Rule 14 emphasizes the if/when constructions 
as the basic valid sentence patters of cooperation-type. 

Consider the list of rules for quality requirements. Rule 1 shows how the quantitative 
value of quality characteristic can be handled. We presuppose that quantor modifier of 
noun phrase can serve such value. The subject in that sentence becomes the QualityMetric 
element. Rule 2 specifies the qualitative parameters of quality requirement. This part of in- 
vestigations is still in progress and requires the additional intelligence methods to distin- 
guish elements of QAPM-S among the whole variety of modeling concepts, such as ap- 
plying specified ontology patterns, which are able to detect the traditional quality 
characteristics (e.g. defined in ISO standards: availability, accessibility, performance, etc.), 
or word class identification via WordNet-based systems, which could provide the assign- 
ment of terms to needed cluster. 


Table 1 — Part of interpretation rules for functional requirements model 


Ne Rule Description Glossary Example 
n0.corelex — 
1. | nO — thing type thing type.classificatio | Thing type Order 
n 
nO(desc=compound) — thing Order processing 
2; ipe Compound noun Thing type gee 
3. | nO(desc=compound) — attribute | Attribute ot Order item 
; Connection | The man with 
4, | n3, n2.child=p2 — post-modifier | Attribute type the hat 
Abstract 
5. vO(verbclass=copV) mA generalization “is a Connection Se niceaareae 
generalization/aggregation or component type 
“is _part_of” 
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Table 1 (Cont.) 


Ne Rule Description Glossary Example 
A vO(verbclass=possV) > peeacon Connection | Each hard drive 
possession type has a capacity 
The order 
vO(verbclass=psychV | tvag2 | Relation between Counéetion. - | department 
7. | locV | tv3 | sentV | tv2) > verb arguments with ; hej 
argument relation respect to its valency type rae ee 
to the order 
vO(verbclass=1V | tvag2 | tv3 | seine for 
sentV | psychV) — operation Activity/Action is P 
: Operation each ordered 
8. | type; D3 subject Actor, 13 object | executed by agent- hecks j 
2 | sentence(type=inf) subject pe Pent enes us 
P : availability on 
— parameters 
stock 
v0(verbclass=i1V | tvag2 | tv3 | 
Sony, | peyeny, temp perl) Completion of Cooperation Pe ccpar nen 
9. | condition; n3 | p2 | activity aoe has checked the 
sentence(type=inf) order,... 
— involvedTypes 
vO(verbclass=tv2) — condition, me Cooperation | Payment is 
10. £3 —rinvolvedTypes Post or precondition type Fesded 
v0(verbclass=locV | possV | Cooperation All ace os 
11. oe Property/state order are in 
copV) — condition type 
stock 
iz ia Cooperation | The window 
12.| vO(verbclass=eV) — condition | Event type broke 
13.] vO(mode=pass) — condition Passive construction Sooper anon |i DHE ances 
: P typt ordered 
If each ordered 
<con0> <n3> <v0> [<n3> | Cee 
s stock, then 
14 <p2> | <sentence>], <adv2> ieCee ean eeaeton Cooperation idee 
“| <n3> <v0> [<n3> | <p2> | type 
<sentence>] department 
relates that item 
to the order. 
Table 2 — Part of interpretation rules for quality requirements model 
Ne Rule Description Element Example 
The response 
<N3 subject? <vO(verbclass=copV)> time to the stock 
1 <N3 objects D3 opject-Child=q2 — N3 subject | The value of Quality | items 
* | -QualityMetric, quality parameter | Metric | replenishment is 
QualityInContext.value=q2 below 2 
seconds. 
<N3 subject” <v0.verbclass=cop V> a2 The characteristic Ouality | Adeeedibiltty is 
2. | — D3 subject -QualityMetric, of quality Matic. | heheceaoea ble 
QualityInContext.description=a2 parameter § P 
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sentence 
£ 4 
: = 
——_--- Sao 
st v. Ped 3 
det 0 po 3 
bee betve | {tose hemp] 4 
4 ak == an 
no "0 ae 4 2 
a ee oe is a 
a a TS 4 free Ipenessi] p 
pom bp | foosaton Ip ote} 4 
ra oe Sr be 
ray ay 
The order department for each ordered \ item checks ts avadabdity on stock 
sentence 
be beter] 
2 3 
aaa & 
con) n3 I 
fm bo] [ 
Bre fosere | 0} 
; ao 
7 fpenesat] | : 
| L wnt 
A be N re 
| iN brs hvo.] 
| \/e ; 
f the | order the department checks the payment 
a 
\ 7 
ld# [Name classification | Quantity est. examplds Source 
D1 |Order prgcessing services servée 1 $1, $2 
D2 |Authorizqtion services se~ice 2 t $3, S4 \ 
D3 [Order hing 1000 $1, $8, $10, $13-S15 
D4 [Order ade Attribute 365 2008-01 de oe \ 
thing 100000 1 ton of again 
D6 |Order department organizatio~ 7 v 
person P ct 
D8 |Payment thing c-id# name T, 5 5 FA ks 
D9 [Bookkeeping departnent organizatio p-id# involved thing-type LEM 
C001 containment podla S-order contains 
poolb gritem is contgi 
involved tila -types requi-rements 
assification|— stp a 
i name 
order processing service é 
order department Ca. 
order item { para 
04 |relateitems |s 
to order ° Pre-condition eration Post-condition aan 
Id# |. involved P involved 
id# Name ce 
types types 
- ET Gt order D3, Order | O7 check Dg, [es payment is D8, $1, 
07 | authori Si comes in payment bookkeeping authorized Payment_| S5 
payment department 
C3 payment is D8, 
O8 |authenticate |s not Payment 
uset o _ authorized 
in parallel 
o1 check all D6, order C4 all articles D3, Order | S1, 
items department are in stock $2 
D5, Order Alternate 
cs not all D3, Order 
E2 |C3 paymentis | D8, O05 Reject order Dg, Cc 
not Payment bookkeeping 
authorized department 
D3, Order “7 


‘ . eo 
Figure 4 — Interpretation process 


Conclusions 


As a result of applying the proposed technique, QAPM-S representation of the set of 
services and qualities of interest (and, optionally, some of their usage contexts) could be 
obtained. This representation serves as a.scope for subsequent quality simulation and 
assessment activities. Automating the tas 4 Gefining the ISAREAD-S scope reduces the 
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up-front costs for applying ISAREAD-S framework. Reducing these costs can be conside- 
red an important step in the direction of increasing the feasibility of its deployment. 
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B.A. ITexoeuoe, M.O. Bastcenoe 

Bukopuectanna 3acodis 06po6Kn NpupoAHO! MOBH JIA BHSHAYeHHA OGO1acTi 3acTOCyBaHHA 
KOPHCTYBaAWbKOLO OWIHIOBAHHA 3MO/JeJIbOBaHO! AKOCTI OOCIYrOByBaHHA 

CraTta TIpHcBaA4eHa BU3HAYeHHIO OOACTi IPOBEACHHA JOCIPKeHb, MOB’ A3aHHX 3 TKIIOUCHHAM 3alliKaBJIeHHx 
ocid WO Mpolecy po3poOKu nporpaMHoro 3a0e3MedeHHA Yepe3 OIHIOBAHHA CIIPHMMaHOi AKOCTI CepBic- 
OpicHTOBaHHX CHCTeM B KOHTCKCTI ixHbOrO BUKOPHCTaHHA, KOJIM MOUaTKOBa CHeuMdikallia CHCTeMN 3ajjaHa 
TIpHpoyHoro Mosoro. ITponoryerbca BUKOpHCTaHHA TeXHOMOrii aHai3y MpHpoAHO! MOB JyId OTPHMaHHsA iH(popMalti 
TIpo OOsacTb 3acTOCyBaHHA 3 Wei cheuM@ikali y topMaTi cieWiabHHX MOJeseM MpezMpoeKTyBaHHA, AKI € 
CYMICHHMH 3 OCHOBHUMH MOJIYJIAMH iMITalliMHOro pileHHA. 


B.A, HTexoeuoe, H.A. Bastcenoe 

Ucnou,30Banne cpexcTB o6paboTKH ecTecTBeHHOrO A3bIKa JIA ONpeeteHHA 0O1acTH WpHMeneHHuAaA 
N1OJIb3OBATEJIbCKOLO OLCHHBAHHA CMOeIMPOBaHHOro Ka4uecTBa OOCyYKHBAHHA 

CraTba MocBalleHa onpeyeienwio oONIacTH MpoBeyeHuA MCCIeqOBaHHM, CBA3AHHBIX C MOAKIIOUCHHeM 
3aMHTepeCOBaHHbIX JIMI, K Mpoleccy pa3pa0oTKH UporpaMMHoro oOecreyeHHA epe3 OLeHHBaHHe 
BOCIPHHUMaeMOroO KayeCTBa CepBHC-OPHeCHTMPOBaHHBIX CHCTEM B KOHTeKCTe HX HCIOJIb3OBAHHA, B CJIyyae, 
eCIM HavasIbHat CieuMPuKaluA CHCTeMbI 3ajj{aHa CCTCCTBCHHBIM A3bIKOM. IIpeqmaraeTca HcrOb30BaHHe 
TeEXHOJIOIMM aHasIv3a CCTECTBCHHOTO AZbIKa JIA MOyYeHuA MHpopMauNu 06 OOacTH MpHMeHeHHA 3 3TOH 
cneuMpukalun B POpMaTe CHeWMaNIbHbIX MOeseH MpeAMpOeKTHPOBaHHA, KOTOPbIC ABJIAIOTCA COBMCCTHMBIMH C 
OCHOBHBIMH MOJYJIAMH HMUTAWMOHHOTO pellleHHa. 


Cmamba nocmynua 6 pedaxyuro 02.07.2010. 
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